Image capturing apparatus and control method thereof

ABSTRACT

An image capturing apparatus determines a scene of a captured image, and, depending on the scene determination result, performs image capture with an expanded dynamic range, or performs a dynamic range contraction process based on the captured image. In the case of executing the dynamic range expansion process, the image capturing apparatus performs image capture at a decreased ISO speed, and performs tone correction for compensating the decrease in ISO speed with respect to the captured image. The image capturing apparatus is thereby capable of performing image capture with a dynamic range that takes into consideration the subject and the scene.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of application Ser. No. 11/694,111,filed Jan. 26, 2010 the entire disclosure of which is herebyincorporated by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image capturing apparatus and acontrol method thereof.

2. Description of the Related Art

Heretofore, digital still cameras and digital video cameras using animage sensor such as a CCD image sensor or a CMOS image sensor have beenwidely used. However, the dynamic range (latitude) of these imagesensors is narrow compared with silver halide film. For this reason,when capturing a high contrast scene, loss of tone detail in lowluminance portions (plugged-up shadows) and loss of tone detail in highluminance portions (blown-out highlights) tend to occur.

Systems capable of controlling the dynamic range automatically have beenproposed in response to such problems.

For example, in Japanese Patent Laid-Open No. 2005-209012, it isproposed, in the case where it is detected from a captured image thatthe main subject is backlit or in a high contrast scene, to specify ablack saturation point and a white saturation point from a histogram ofthe image, and perform tone correction such that the brightness of themain subject is correct.

Also, in Japanese Patent Laid-Open No. 2004-186876, it is proposed toacquire two types of images with different dynamic ranges using an imagesensor disposed with high sensitivity light receiving elements and lowsensitivity light receiving elements, by capturing the same scene withthe light receiving elements having different sensitivities, and combinethese captured images according to a scene analysis result.

There is also a digital camera (Canon EOS 5D Mark II) that has ashooting mode (highlight tone priority mode) for shifting thesensitivity setting range one step higher and suppressing blown-outhighlights in high luminance portions.

However, with the method of Japanese Patent Laid-Open No. 2005-209012,an effect of widening the dynamic range is not obtained, since thepoints at which the sensor becomes saturated do not change, despitethere being an effect of increasing contrast to the eye.

Also, with the method of Japanese Patent Laid-Open No. 2004-186876, aneffect of widening the dynamic range is obtained, but there is theproblem of the cost involved, since a special image sensor disposed withlight receiving elements having different sensitivities needs to beused.

Also, with the digital camera having the tone priority mode, to obtainan image capture result in which blown-out highlights in high luminanceportions are suppressed, the user needs to explicitly set the tonepriority mode before performing image capture. Also, depending on thescene, there may be cases where the effect of the tone priority mode isnot really noticeable or where it would be better to prioritize otherconditions rather than suppressing blown-out highlights in highluminance portions when performing image capture. For example, in thecase where the subject is a landscape, desirably there are no blown-outhighlights in high luminance portions, but in the case where the subjectis a person, desirably the brightness of the person's face is correct,even if the background is slightly blown out. In other words, to obtainan appropriate image capture result by making use of the tone prioritymode, the user is required to be able to discriminate scenesappropriately, in addition to setting the tone priority mode.

SUMMARY OF THE INVENTION

The present invention has been made in consideration of such problemswith the conventional art. The present invention provides an imagecapturing apparatus and a control method thereof that are capable ofperforming image capture in a dynamic range that takes into account thesubject and the scene.

According to an aspect of the present invention, there is provided animage capturing apparatus comprising: scene determining unit arranged toperform scene determination, based on information regarding a movingimage captured prior to the capture of a still image; and control unitarranged to control the image capturing apparatus to capture an image,and to execute, based an output from the scene determination unit, oneof (a) a dynamic range expansion process for capturing a still image ata decreased ISO speed, and applying, with respect to a captured stillimage, tone correction for compensating the decrease in ISO speed, or(b) a dynamic range contraction process for capturing a still image, andapplying, with respect to a captured still image, tone correction forincreasing the brightness of a low luminance portion of the capturedstill image, based on a characteristic of a high luminance portion ofthe moving image or the captured still image.

According to an aspect of the present invention, there is provided acontrol method of an image capturing apparatus, comprising: a scenedetermining step of performing scene determination, based on a movingimage captured prior to the capture of a still image; and a control stepof controlling an image capturing operation, wherein in the controlstep, based on a result of the scene determination in the scenedetermining step, one of (a) a dynamic range expansion process forcapturing a still image at a decreased ISO speed, and applying, withrespect to a captured still image, tone correction for compensating thedecrease in ISO speed, or (b) a dynamic range contraction process forcapturing a still image, and applying, with respect to a captured stillimage, tone correction for increasing the brightness of a low luminanceportion of the captured still image, based on a characteristic of a highluminance portion of the moving image or the captured still image, isexecuted.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing an example configuration of an imagecapturing apparatus according to an embodiment of the present invention.

FIGS. 2A to 2D schematically show the process of detecting a face froman original image.

FIG. 3 is a flowchart illustrating a face detection operation in a facedetection circuit of the image capturing apparatus according to theembodiment of the present invention.

FIG. 4 is a chromaticity diagram showing representative colors in CIEL*a*b* color space.

FIG. 5A shows example coefficients of a two-dimensional high-pass filterused by the face detection circuit of the image capturing apparatusaccording to the embodiment of the present invention.

FIG. 5B shows example region partitioning when determining whether thescene is a backlight scene, in the image capturing apparatus accordingto the embodiment of the present invention.

FIG. 6 is a block diagram showing an example configuration of an AFEcircuit of the image capturing apparatus according to the embodiment ofthe present invention.

FIG. 7 schematically shows example histograms created by a histogramcreation circuit of the image capturing apparatus according to theembodiment of the present invention.

FIG. 8 is a block diagram showing the flow of signal processing duringstanding by to capture a still image, in the image capturing apparatusaccording to the embodiment of the present invention.

FIG. 9 is a flowchart illustrating an operation for determining anamount of dynamic range (D-range) expansion in the image capturingapparatus according to the embodiment of the present invention.

FIGS. 10A to 10D show specific examples of final amounts of D-rangeexpansion determined according to the size of an amount of D-rangeexpansion for a face region and an amount of D-range expansion amountfor an entire image, in the image capturing apparatus according to theembodiment of the present invention.

FIG. 11A represents a conceptual view of the D-range in the imagecapturing apparatus according to the embodiment of the presentinvention.

FIG. 11B shows an example of the relation between AE target value,saturation signal value and D-range, in the image capturing apparatusaccording to the embodiment of the present invention.

FIGS. 12A and 12B show an example setting of tone characteristics in asignal processing circuit of the image capturing apparatus according tothe embodiment of the present invention.

FIGS. 13A and 13B are flowcharts illustrating operation of the actualimage capturing process, in the image capturing apparatus according tothe embodiment of the present invention.

FIGS. 14A and 14B show an example of tone characteristics selected whenthe scene is determined to be a night scene, in the image capturingapparatus according to the embodiment of the present invention.

FIG. 15 is a flowchart illustrating operation of determining an amountof dynamic range contraction (D−), in the image capturing apparatusaccording to the embodiment of the present invention.

FIG. 16 shows an example of the relation between final amounts of gainincrease that depend on a final value of a gain value Gain_yhp computedat S504 of FIG. 15 and ISO speed, in the image capturing apparatusaccording to the embodiment of the present invention.

FIGS. 17A and 17B show the relation between amount of gain increase andtone characteristic setting value, in the image capturing apparatusaccording to the embodiment of the present invention.

FIG. 18 shows an example of the relation between the setting of exposurecontrol, dynamic range expansion process (D+), dynamic range contractionprocess (D−), and tone characteristics for a night scene, according to ascene determination result, in the image capturing apparatus accordingto the embodiment of the present invention.

DESCRIPTION OF THE EMBODIMENTS

Preferred embodiments of the present invention will now be described indetail in accordance with the accompanying drawings.

FIG. 1 is a block diagram showing an example configuration of an imagecapturing apparatus according to an embodiment of the present invention.The image capturing apparatus of the present embodiment encompassesarbitrary apparatuses having a function of capturing images using animage sensor. Such apparatuses include mobile telephones, PDAs andpersonal computers that incorporate or are connected to a camera, aswell as digital still cameras and digital video cameras.

In FIG. 1, an operation unit 112 includes buttons, switches or the like,and is used by a user for giving instructions and configuring settingswith respect to the image capturing apparatus. The operation unit 112also includes a shutter button, and, in the present embodiment, is ableto detect a half stoke state and a full stroke state of the shutterbutton.

A system controller 107 perceives a half stroke state of the shutterbutton as an image capture preparation instruction, and a full strokestate of the shutter button as an image capture start instruction. Thesystem controller 107, for example, includes a CPU, a ROM and a RAM, andcontrols the overall operation of the image capturing apparatus as aresult of the CPU executing a program stored in the ROM, using the RAM.

An orientation sensor 133 detects an orientation of the image capturingapparatus, and outputs a detection result to the system controller 107.The system controller 107 determines whether the orientation of theimage capturing apparatus is vertical or horizontal, based on the outputfrom the orientation sensor 133. Note that the orientation sensor 133may determine whether the orientation of the image capturing apparatusis vertical or horizontal, and output the determination result to thesystem controller 107.

A lens apparatus 200 has a group of lens including a focus lens, adriving apparatus that drives the focus lens, an aperture and amechanical shutter, and operates under the control of the systemcontroller 107.

An image sensor 101 is a photoelectric conversion element such as a CCDimage sensor or a CMOS image sensor. An analog front end (AFE) circuit150 performs gain adjustment, A/D conversion and the like on analogimage signals output from the image sensor 101, and outputs the resultas digital image signals. The AFE circuit 150 will be discussed indetail later.

A buffer memory 103 temporarily stores the digital image signals outputby the AFE circuit 150.

A compression/decompression circuit 104 encodes captured image data toan image file (e.g., JPEG file) format for recording, and decodes imagefiles read from a recording medium 106.

A recording apparatus 105 performs reading and writing of data under thecontrol of the system controller 107, with respect to a recording medium106 such as a built-in memory or a removable memory card.

A display control circuit 108 controls a display operation with respectto a display unit 110 that includes a display device such as an LCD,under the control of the system controller 107.

A D/A converter 109 converts digital image signals for display that areoutput by the display control circuit 108 to analog image signals thatcan be displayed by the display unit 110.

The display unit 110 performs display of GUI windows for the user toconfigure various settings and give instructions with respect to theimage capturing apparatus, and display of various types of informationrelating to the image capturing apparatus, and the like, as well asperforming display of captured images. Also, the display unit 110 can becaused to function as an electronic viewfinder (EVF), by sequentiallydisplaying, on the display unit 110, images captured continuously. Thissequential image capture/display operation for causing the display unit110 to function as an EVF is also called “through-the-lens view” or“live view”. This sequential image capturing operation is substantivelyequivalent to an image capturing operation for moving images, and imagescaptured for EVF use and displayed are in effect moving images.

A face detection circuit 120 performs face detection as an examplemethod for detecting a person from a captured image. The face detectioncircuit 120 performs the face detection process on image data in YUVformat or RAW format stored in the buffer memory 103, and outputs a facedetection result that includes the size and position of the face regionin the image to a histogram creation circuit 130.

There are no particular restrictions on the face detection method usedby the face detection circuit 120, and an arbitrary known method can beapplied. As for known face detection techniques, many different methodhave been proposed, including methods based on learning that use neuralnetworks and the like, methods for retrieving sites with characteristicshapes such as the eyes, nose and mouth from an image using templatematching, and viewing these sites as a face the higher the degree ofsimilarity, and methods for detecting the amount of image features suchas color of skin and shape of eyes, and using statistical analysis onthese image features. A plurality of these methods can also be combinedto improve the accuracy of face detection. Specific examples include amethod of face detection using wavelet transform and image featureamounts disclosed in Japanese Patent Laid-Open No. 2002-251380.

Here, a specific example of a face detection operation of the facedetection circuit 120 will be described, with reference to FIG. 2A toFIG. 5A.

FIGS. 2A to 2D schematically show the process of detecting a face froman original image, with FIG. 2A showing the original image.

FIG. 3 is a flowchart illustrating a face detection operation of theface detection circuit 120.

At S101, the face detection circuit 120 extracts a skin color regionfrom the original image. FIG. 4 is a chromaticity diagram showingrepresentative colors in CIE L*a*b* color space, and the ellipsoid inthe middle is the region most likely to be skin color.

The face detection circuit 120 converts the RGB original image data toL*a*b* format with a known method, and extracts a skin color regioncomposed of pixels having the chromaticity of the region shown by theellipsoid in FIG. 4. FIG. 2B schematically shows the skin color regionextracted from the original image.

Next, at S102, the face detection circuit 120 extracts a high frequencycomponent from the extracted skin color region. Specifically, the facedetection circuit 120 applies a high-pass filter to the skin colorregion. FIG. 5A shows example coefficients of a two-dimensionalhigh-pass filter. An example image obtained by applying the high-passfilter to the image in FIG. 2B is shown in FIG. 2C.

At S103, the face detection circuit 120 performs template matching onthe image after application of the high-pass filter, using an eyetemplate such as shown in FIG. 2D, and detects the eyes in the image.

At S104, the face detection circuit 120 determines the face region basedon the positional relation of the eye region detected at S103, and thelike, and derives a face detection result that includes the position andsize of the face region.

Returning to FIG. 1, the histogram creation circuit 130 acquires thedetection result of the face region from the face detection circuit 120,and creates a histogram relating to the luminance values of pixelsincluded in the face region. The histogram creation circuit 130 can alsocreate a histogram of the luminance values of pixels that are included,for each of a plurality of partial regions obtained by partitioning theimage. Created histograms are stored in the buffer memory 103.

Also, in the case of acquiring color information for each partialregion, color information for each partial region is acquired with acolor evaluation value acquisition circuit 132, with respect to acontracted image obtained by contracting one screen worth of images forimage analysis stored in the buffer memory 103 with a resizing circuit131. The color information may be color evaluation values such assaturation, hue and luminance, for example.

A signal processing circuit 140 applies signal processing to image datastored in the buffer memory 103, in accordance with signal processingparameters (white balance correction coefficients, tone characteristicparameters, etc.) set by the system controller 107. The signalprocessing circuit 140 then generates YUV image data, and again storesthe image data in the buffer memory 103.

As will be discussed later, the image capturing apparatus of the presentembodiment realizes dynamic range control, by sensitivity adjustment(gain adjustment) in the AFE circuit 150 and tone characteristiccorrection in the signal processing circuit 140.

FIG. 6 is a block diagram showing an example configuration of the AFEcircuit 150.

A clamp circuit 151 clamps signals output from the image sensor 101 to areference black level, such that the output values when the sensor isblocked or the output values of a reference voltage region of the sensorwill be zero.

A CDS gain circuit 152 applies a CDS gain (analog gain) to clampedsignals. The CDS gain applied by a generic AFE circuit has a discretevalue such as 0, 3 or 6 [dB].

Signals to which the analog gain has been applied are converted todigital data by an A/D converter 153. Next, a variable gain amplifier(VGA) gain circuit 154 applies a VGA gain to the digital data. In thepresent embodiment, the value of the VGA gain is adjustable in 0.125 dBincrements in a range of 0 to 36 dB, for example. Signals to which theVGA gain has been applied are output to the buffer memory 103 afterbeing clipped to a prescribed number of bits by a clipping circuit 155.

In the present embodiment, setting of ISO speed in the image capturingapparatus is realized by the system controller 107 controlling thevalues of the CDS gain and the VGA gain applied by the AFE circuit 150.

Also, sensitivity adjustment for correcting variability in thesensitivity characteristics of the image sensor 101 can be performed, byirradiating the image capturing apparatus with light of a referencelight amount, and controlling the CDS gain and VGA gain such that aconstant luminance signal value is output from the image capturingapparatus.

In the present embodiment, sensitivity setting is performed by combiningthe CDS gain and the VGA gain. For example, in the case where a gain of6 dB in total is set for low sensitivity, the CDS gain is set to 3 dBand the VGA gain is set to 3 dB, for example. Also, in the case where again of 24 dB is set for high sensitivity, the CDS gain is set to 6 dBand the VGA gain is set to 18 dB, for example. As abovementioned, sincefine setting cannot generally be performed with the CDS gain, anapproximate gain is set with the upstream CDS gain, and the VGA gain iscontrolled in order to perform subtle sensitivity control of asensitivity adjustment portion or the like.

Generally, a gain circuit also amplifies the noise component at the sametime as the signal component. For this reason, in order to suppressamplification of the superimposed noise component with an analogcircuit, the upstream CDS gain preferably is set as high as possible, inthe combination of the CDS gain and the VGA gain that is able to realizethe total gain amount. This setting also enables the effect of beingable to effectively maximize the quantization accuracy of the A/Dconverter 153 to be realized.

Next, operation when image capture is performed in an image capturingapparatus having the abovementioned configuration will be described.

The image capturing apparatus of the present embodiment, during standingby in a shooting mode where an image capture preparation instruction oran image capture start instruction has not been input, captures movingimages and causes the display unit 110 to function as an EVF. That is,the system controller 107 executes a process of capturing imagescontinuously at a prescribed rate (e.g., 30 frames/sec.), generatingdisplay images from the captured images, and causing the display unit110 to display the generated display images.

In the case where execution of face detection is set, the face detectioncircuit 120 performs face detection on the display images (hereafter,also referred to as EVF images), and outputs the detection results tothe system controller 107. The system controller 107 then instructs thedisplay control circuit 108 to superimpose a face frame for presentingthe detected face region to the user on the EVF images, together withthe position information of the face region.

The face detection results are also supplied to the histogram creationcircuit 130, and the histogram creation circuit 130 creates histogramsfrom the pixels included in the face region within the EVF images. Also,the histogram creation circuit 130 creates histograms for each of aplurality of regions into which the entire image is partitioned. Createdhistograms are stored in the buffer memory 103.

FIG. 7 schematically shows example histograms created by the histogramcreation circuit 130 in the present embodiment.

FIG. 7 shows the creation of a histogram 73 for each partial region 71obtained by quartering an entire image both horizontally and vertically,and a histogram 74 for a face region 72. Note that the histograms 73 and74 in FIG. 7 are cumulative histograms. Note also that the facial regionmay be excluded when creating histograms for the partial regions. Thisenables histograms to be created for the face region and for regionsother than the face region (background).

In the present embodiment, a luminance value YHi with a frequency of 80%in the cumulative histogram 73 of the partial regions 71, and aluminance value YHiFace with a frequency of 90% in the cumulativehistogram 74 of the face region 72 are used in evaluating the blown-outhighlights region of a captured image, such as will be discussed later.

When an image capture start instruction is input by the user pressingthe shutter button a full stroke, the system controller 107 performs animage capturing operation based on the processing results of autoexposure control (AE), auto focus detection (AF) and the like.Specifically, the system controller 107 performs image capture bycontrolling the focal position and aperture of the lens apparatus 200,the mechanical shutter, the image sensor 101, and, further, a flash (notshown) as required.

Analog image signals output from the image sensor 101 are stored in thebuffer memory 103 as digital image data, via the abovementioned AFEcircuit 150. The signal processing circuit 140 performs processing onthis digital image data, in accordance with various signal processingparameters set by the system controller 107, generates YUV image data,and again stores the generated image data in the buffer memory 103.

Image data processed by the signal processing circuit 140 is encoded ina JPEG file, for example, by the compression/decompression circuit 104,and recorded to the recording medium 106 by the recording apparatus 105.

Also, the display control circuit 108 generates a display image from theYUV image data stored in the buffer memory 103, and causes the displayunit 110 to display the display image as a quick review image via theD/A converter 109.

FIG. 8 is a block diagram showing the flow of signal processing duringstanding by to capture a still image, in the image capturing apparatusaccording to the present embodiment. As abovementioned, during standingby to capture a still image, the image capturing apparatus of thepresent embodiment performs continuous image capture and display forcausing the display unit 110 to function as an EVF.

Analog image signals output from the image sensor 101 are gain adjusted(sensitivity adjusted) and digitized by the AFE circuit 150. So-calleddeveloping processes such as pixel interpolation and white balancecorrection are performed by the signal processing circuit 140 on thisRAW image data, and YUV digital image data is generated.

This digital image data is stored in a display region (commonly called aVRAM or a display buffer) 103 a of the buffer memory 103, and is outputto the display unit 110 via the display control circuit 108 and the D/Aconverter 109.

On the other hand, the digital image data generated by the signalprocessing circuit 140 is also stored in an image analysis region (imageanalysis buffer) 103 b of the buffer memory 103. The image data storedin the image analysis buffer 103 b is used in face detection by the facedetection circuit 120 and histogram creation by the histogram creationcircuit 130. Note that not all of the EVF image data needs to be storedin the image analysis buffer 103 b, and only the portion of EVF imagedata that depends on the cycle of performing face detection andhistogram creation is stored.

The face detection circuit 120 performs face detection on image datastored in the image analysis buffer 103 b, and if a face is detected,outputs a face detection result that includes information (e.g.,position and size) that enables the face region to be specified.

The histogram creation circuit 130 creates a histogram, based on imagedata stored in the image analysis buffer 103 b and the face detectionresult from the face detection circuit 120. As abovementioned, thehistogram creation circuit 130 is able to create a face region histogram130 a for the face region, and a partitioned region histogram 130 b foreach region image obtained by partitioning the entire image.

The partitioned region histogram 130 b can also be created for eachregion image including the face region, or derived for each region imageexcluding the face region, with regard to the entire image. The formeris easily processed, but the latter is preferable in order to accuratelydetect whether the region with blown-out highlights is a face region ora background region.

As abovementioned, image capture is continuously repeated for theduration that the display unit 110 is being caused to function as anEVF, with the VRAM being rewritten in quick succession. Generally, thetime needed for face detection and histogram creation is longer than thedisplay cycle of EVF images (e.g., 1/30 sec.). For this reason, in thepresent embodiment, the image analysis buffer 103 b is provided inaddition to the VRAM, and the image analysis buffer 103 b is not updateduntil face detection and histogram creation on image data stored in theimage analysis buffer 103 b has ended.

As a result of this configuration, it is possible to perform facedetection and histogram creation on the same EVF image, and to performimage analysis easily and with a high degree of accuracy. Of course,there is no harm in executing face detection and histogram creation oneach frame of the EVF images if this is possible, but there is no needto perform face detection and histogram creation on each frame, since itis generally unlikely that the captured scene will change greatly fromframe-to-frame. Thus, the load on the system controller 107 can bereduced.

Also, image data stored in the image analysis buffer 103 b is contractedby the resizing circuit 131, and color evaluation values are acquired bythe color evaluation value acquisition circuit 132.

For example, assume that image data stored in the image analysis buffer103 b corresponds to a VGA-size image. The resizing circuit 131 firstlygenerates YUV 4:2:2 image data of 64 horizontal pixels by 48 verticalpixels, for example, from YUV 4:2:2 image data of 640 horizontal pixelsby 480 vertical pixels. Subsequently, the resizing circuit 131 generatesYUV 4:4:4 image data of 32 horizontal pixels by 24 vertical pixels byaveraging Y with blocks composed of 4 pixels (2 pixels in horizontaldirection and 2 pixels in vertical direction), and averaging UV with 2pixels in the vertical direction.

Note that as for the resizing method in the resizing circuit 131, othermethods may be used, such as averaging in prescribed units of aplurality of pixels, simple resampling, thinning by linearinterpolation, and bicubic interpolation.

The color evaluation value acquisition circuit 132 computes luminanceinformation Y, hue information H and saturation information C as colorevaluation values for each block, with respect to the contracted YUV4:4:4 image data of 32 horizontal pixels by 24 vertical pixels generatedby the resizing circuit 131.

The color evaluation value acquisition circuit 132 is able to computethe color evaluation values using the following equations, for example.Y=YC=√{square root over (U ² +V ²)}H=tan⁻¹(U/V)

Of course, the color evaluation values may be evaluation values inCIELab color space, or evaluation values in another color space. Also,computation of color evaluation values may be performed using a mathlibrary for performing the above arithmetic operations or may bysimulated by referring to a lookup table prepared in advance.

Luminance information, hue information and saturation information canthereby be acquired as color evaluation values for each block obtainedby partitioning the entire image into blocks of 32 horizontal pixels by24 vertical pixels.

FIG. 9 is a flowchart illustrating an operation for determining anamount of dynamic range expansion (D+) in the image capturing apparatusof the present embodiment.

In the present embodiment, the amount of blown-out highlights incaptured images is derived based on face detection results and histogramcreation results with respect to EVF images, and the dynamic rangeexpansion amount is determined according to the amount of blown-outhighlights. The exposure and sensitivity for when performing actualimage capture are adjusted using the amount of dynamic range expansiondetermined in advance using EVF images.

At S201, the system controller 107, as abovementioned, continuouslyperforms image capture, causes EVF images to be generated, and causesthe EVF images to be stored in the display buffer 103 a sequentiallyfrom the signal processing circuit 140. The system controller 107 alsocauses the EVF images to be stored in the image analysis region (imageanalysis buffer) 103 b of the buffer memory 103 at a predeterminedcycle.

At S202, the face detection circuit 120 performs face detection on anEVF image stored in the image analysis buffer 103 b.

In the case where face detection is successful (S203: YES), thehistogram creation circuit 130 creates a face region histogram from theface region of the EVF image based on the face detection result from theface detection circuit 120 (S204).

At S205, the system controller 107 computes the amount of blown-outhighlights of the face region from the face region histogram.

If face detection is not successful (S203: NO), or after computation ofthe amount of blown-out highlights of the face region, the histogramcreation circuit 130, at S206, creates a partitioned region histogramfor each region image obtained by partitioning the entire EVF image.Here, as an example, assume that the histogram creation circuit 130creates a partitioned region histogram for each of the 16 partitionedregions obtained by quartering an entire EVF image horizontally andvertically.

At S207, the system controller 107 computes the amount of blown-outhighlights of the entire EVF image from the partitioned regionhistograms.

At S208, the system controller 107 determines the amount of dynamicrange expansion, based on at least the amount of blown-out highlightsfor the entire image derived at S207.

Next, a specific example of the process of computing the amount ofblown-out highlights performed by the system controller 107 at S205 andS207 of FIG. 9 will be described.

Firstly, the process of computing the amount of blown-out highlights ofthe face region will be described.

At S205, the system controller 107 computes, as the amount of blown-outhighlights of the face region, a luminance value YHiFace at which thecumulative frequency of a cumulative histogram is a prescribed value(90% in the FIG. 7 example), from the face region histogram created atS204.

At S208, the system controller 107 then determines the amount of dynamicrange expansion for the face region (D+(face)), according to therelative size relation between the value of the amount of blown-outhighlights of the face region YHiFace and predetermined thresholdvalues.

Specifically, when the predetermined threshold values are the threesteps of THHiFace, THMidFace and THLowFace in descending order, forexample, the amount of dynamic range expansion for the face region willbe determined as follows:D+(face)=Correction Level 1 Step(if YHiFace>THHiFace)D+(face)=Correction Level 2/3 Steps(if THHiFace≧YHiFace>THMidFace)D+(face)=Correction Level 1/3 Steps(if THMidFace≧YHiFace>THLowFace)D+(face)=Correction Level 0 Steps(if THLowFace≧YHiFace)

Also, at S206, the system controller 107 computes the luminance valueYHi_n (n=1 to several partitions; 16 in FIG. 7 example) at which thecumulative frequency of a cumulative histogram is a prescribed value(80% in FIG. 7 example), as the amount of blown-out highlights of thepartial region, with regard to each partitioned region histogram createdat S206.

At S207, the system controller 107 counts the number of regions YH_BNumin which the luminance value YHi_n exceeds the predetermined thresholdvalue Y_B_Th. The system controller 107 then determines the amount ofdynamic range expansion for the entire image (D+(background)), accordingto the relative size relation between the number of regions YH_BNum andpredetermined threshold values.

Specifically, when the predetermined threshold values are ThYH_BNum6 toThYH_BNum0 in descending order, for example, the amount of dynamic rangeexpansion for the entire image will be determined as follows:D+(background)=Correction Level 6/3 Steps(if YH _(—) BNum>ThYH _(—)BNum6)D+(background)=Correction Level 5/3 Steps(if ThYH _(—) BNum6≧YH _(—)BNum>ThYH _(—) BNum5)D+(background)=Correction Level 4/3 Steps(if ThYH _(—) BNum5≧YH _(—)BNum>ThYH _(—) BNum4)D+(background)=Correction Level 3/3 Steps(if ThYH _(—) BNum4≧YH _(—)BNum>ThYH _(—) BNum3)D+(background)=Correction Level 2/3 Steps(if ThYH _(—) BNum3≧YH _(—)BNum>ThYH _(—) BNum2)D+(background)=Correction Level 1/3 Steps(if ThYH _(—) BNum2≧YH _(—)BNum≧ThYH _(—) BNum1)D+(background)=Correction Level 0 Steps(if ThYH _(—) BNum1≧YH _(—) BNum)

In other words, the system controller 107 determines the amount ofdynamic range expansion to be larger the greater the area of theblown-out highlights region in the image.

Note that the method of determining the blown-out highlights region isnot limited to the method using cumulative histograms described here,and other arbitrary methods can be used.

At S208, the system controller 107 determines the final amount ofdynamic range expansion. Here, in the case where face detection issuccessful, the system controller 107 determines the final amount ofdynamic range expansion by comparing the amounts of dynamic rangeexpansion determined at S205 and S207. For example, the systemcontroller 107 is able to determine the larger amount of expansion asthe final amount of dynamic range expansion, out of the amount ofdynamic range expansion for the face region and the amount of dynamicrange expansion for the entire image.

Alternatively, the system controller 107 may determine one of the amountof dynamic range expansion for the face region and the amount of dynamicrange expansion for the entire image as the final amount of dynamicrange expansion according to a shooting mode set by a mode dial or thelike included in the operation unit 112. For example, in the case of ashooting mode for capturing a person (e.g., Portrait mode), the amountof dynamic range expansion for the face region can be determined as thefinal amount of dynamic range expansion, whereas in the case of ashooting mode for capturing a landscape (e.g., Landscape mode), theamount of dynamic range expansion for the entire image or backgroundregion can be determined as the final amount of dynamic range expansion.

Also, the amount of dynamic range expansion may be controlled accordingto the scene determination result. For example, the amount of dynamicrange expansion of the background region may be expanded to 2 steps (6/3steps) when the scene is determined to be a backlight scene, andotherwise stopped at 1 step (3/3 steps).

Also, a method other than one of the amount of dynamic range expansionfor the face region and the amount of dynamic range expansion for theentire image being selected and determined as the final amount ofdynamic range expansion may be employed. For example, the final amountof dynamic range expansion can also be determined according to the sizeof the amount of dynamic range expansion for the face region and theamount of dynamic range expansion for the entire image.

FIGS. 10A to 10D show specific examples of final amounts of dynamicrange expansion determined according to the size of the amount ofdynamic range expansion for the face region and the amount of dynamicrange expansion for the entire image.

With the examples shown FIGS. 10A to 10C, the value of the final amountof dynamic range expansion determined is changed for each shooting mode.For example, in the case where the amount of dynamic range expansion forthe face region is 1/3 steps, and the amount of dynamic range expansionfor the entire image is 0/3 steps, the final amount of dynamic rangeexpansion will be 1/3 steps in Auto mode (FIG. 10A) and Portrait mode(FIG. 10C), and 0/3 steps in Landscape mode (FIG. 10B).

Note that in the examples shown in FIGS. 10A to 10C, the amount ofdynamic range expansion for the background image is up to 1 step (3/3steps).

FIG. 10D shows an example in which the range of the final amount ofdynamic range expansion determined according to the scene determinationresult is changed. In this example, if the scene is determined to be abacklight scene, the amount of dynamic range expansion for the faceregion remains up to 3/3 steps, but the amount of dynamic rangeexpansion for the background region is extended up to 6/3 steps.

The system controller 107 stores the amounts of dynamic range expansiondetermined as described above in the buffer memory 103, and refers tothe stored amounts when performing image capture. The operation ofdetermining the amounts of dynamic range expansion can be performedevery fixed number of frames of EVF images or every fixed period oftime, for example, when in standby, and the latest amounts of dynamicrange expansion are stored in the buffer memory 103.

Scene Determination

Next, scene determination operations in the image capturing apparatus ofthe present embodiment will be described.

While there are no particular restrictions on the type of scenedetermined by the image capturing apparatus of the present embodiment,operations for determining the presence of a person, a night scene, ablue sky scene, a sunset scene, a backlight scene, and macro shootingwill be described here as examples.

In the present embodiment, scene determination is executed by the systemcontroller 107, based on the following:

exposure information such as subject luminance (By value) obtained froma photometric sensor or a captured image

subject distance information obtained with the auto focus operation

color temperature information obtained by white balance control

color evaluation values for each block obtained by the color evaluationvalue acquisition circuit 132

histograms obtained by the histogram creation circuit 130

orientation information of the image sensor obtained from the output ofthe orientation sensor 133

Night Scene Determination

The scene is determined to be a night scene if all of the followingconditions (A1) to (A3) are satisfied:

(A1) By value is less than or equal to 0 [EV].

(A2) Average luminance value of the entire image computed from the colorevaluation values for each block is less than or equal to a prescribedvalue.

(A3) Percentage of blocks within a prescribed range from the top of theimage whose average luminance and average saturation obtained from thecolor evaluation values satisfy the following conditions is greater thanor equal to a prescribed percentage:

average luminance less than or equal to prescribed value Y_Th_night

average saturation less than or equal to prescribed value C_Th_night

Note that the top and bottom of the image can be determined based on theoutput of the orientation sensor 133.

Also, the conditions may be changed depending on the range of By values.

Further, the mounting of a tripod may be taken into consideration. Themounting of a tripod can, in the case of the image capturing apparatusincorporating a vibration detection sensor for correcting camera shake,be judged from the output of the vibration detection sensor. Themounting of a tripod may also be determined based on the movement of thesubject within a captured image. In the case of it being determined thata tripod is mounted, the scene may be determined to be a tripod nightscene, as distinct from a normal night scene.

In the case of a tripod night scene without people, for example,exposure control such as lowering the ISO speed and making a longexposure may be performed, since there is little concern about camerashake or subject blur.

Also, in the case of a tripod night scene with people, desirably controlof exposure and flash metering is performed, such that the brightness ofthe face region and the background are optimized.

Also, a better image is obtained by flattening the tone characteristicsof dark portions (reducing to a low tone level), since the tendency isfor shadows to stand out and noise to increase as a result of making along exposure.

Blue Sky Scene Determination

The scene is determined to be a blue sky scene if all of the followingconditions (B1) to (B4) are satisfied:

(B1) By value is greater than or equal to 5 [EV].

(B2) Subject distance at the time of shooting is greater than or equalto a prescribed value (i.e., not macro shooting).

(B3) Color temperature is in a prescribed color temperature range.

(B4) Percentage of blocks within a prescribed range from the top of theimage whose average luminance, average hue and average saturationobtained from the color evaluation values satisfy the followingconditions is greater than or equal to a prescribed percentage:

average luminance within prescribed range (≧Y_Th_Sky_Low, ≦Y_Th_Sky_Hi)

average hue within prescribed range (≧Hue_Th_Sky_Low, ≦Hue_Th_Sky_Hi)

average saturation within prescribed range (≧C_Th_Sky_Low, ≦C_Th_Sky_Hi)

Also, the conditions may be changed depending on the range of By values.

For example, in the case of a blue sky scene without people, there isconsidered to be a high probability of the scene being a landscape shot.For this reason, a better image is obtained when processing such asenhancing saturation for the entire image such as blue and other colorsand increasing contrast is performed.

Also, in the case of a blue sky scene with people, there is a highprobability of the scene being a landscape snap shot. For this reason,an appropriate image is obtained when enhancement of near skin colorsaturation is restrained while enhancing blue saturation, and AE andtone characteristic settings are performed such that exposure and toneof the face is optimized.

Sunset Scene Determination

The scene is determined to be a sunset scene if all of the followingconditions (C1) to (C7) are satisfied.

(C1) By value is greater than or equal to 7 [EV].

(C2) Subject distance at the time of shooting is greater than or equalto a prescribed value (i.e., not macro shooting).

(C3) Average Luminance value of the entire image computed from the colorevaluation values is less than or equal to a prescribed value.

(C4) Percentage of high luminance blocks with an average luminancegreater than or equal to a prescribed value is greater than or equal toa prescribed percentage.

(C5) Percentage of blocks whose average luminance, average hue andaverage saturation obtained from the color evaluation values satisfy thefollowing conditions is greater than or equal to a prescribedpercentage:

average luminance within prescribed range (≧Y_Th_Sunset_Low,≦Y_Th_Sunset_Hi)

average hue within prescribed range (≧Hue_Th_Sunset_Low,≦Hue_Th_Sunset_Hi)

average saturation within prescribed range (≧C_Th_Sunset_Low,≦C_Th_Sunset_Hi)

(C6) Number of blocks that do not satisfy one of the followingconditions is less than or equal to a prescribed value:

condition (C5) of sunset scene determination

condition (B4) of blue sky scene determination

average saturation less than or equal to prescribed value

average luminance greater than or equal to prescribed value (highluminance block)

average luminance less than or equal to prescribed value (low luminanceblock)

(C7) Histograms of hue and saturation in the color evaluation valueshave dispersions greater than or equal to prescribed values.

Also, the conditions may be changed depending on the range of By values.Smeared regions may also be eliminated based on the color evaluationvalues.

If the scene is determined to be a sunset scene, a better image isobtained when enhancement of the saturation of reds and oranges and WBsetting in the high color temperature direction of clouds and the likeare performed, together with performing AE control to slightlyunderexpose.

Backlight Scene Determination

(1) Case where a Face is Detected

The scene is determined to be a backlight scene with people if thedifference between the average luminance derived from the colorevaluation values of blocks corresponding to the face region and theaverage luminance derived from the color evaluation values of otherblocks is greater than or equal to a prescribed EV value.

(2) Case where a Face is not Detected

If AE control converges for a fixed period of time, the entire image ispartitioned into four types of regions A to D in order from the outside,as shown in FIG. 5B, and the average luminance value for each region iscomputed from the color evaluation values of the blocks corresponding tothe respective regions.

The average luminance value of the region A is compared with therespective average luminance values of the regions B to D, and if adifference greater than or equal to a prescribed EV value is detected,the scene is determined to be a backlight scene without people.

Apart from the method discussed here, luminance information obtained bypartitioning a CCD-RAW image into blocks or histogram information may beused.

The backlight determination in the present embodiment involves detectinga two-dimensional luminance pattern, and differs from detecting thedegree of blown-out highlights of luminance for each block or region,such as performed when detecting blown-out highlights. Accordingly,backlight scene determination and detection of blown-out highlights aredifferent processes.

However, since there is considered to be a high possibility of a scenedetermined to be a backlight scene having tone loss in high luminanceportions, a better image can be obtained by increasing the amount ofdynamic range expansion if a scene is determined to be a backlightscene.

Macro Shooting Determination

Whether or not the scene involves macro shooting can be determined basedon the focused subject distance. The subject distance can be obtainedfrom the position of the focus lens, for example, with reference to apreset correspondence table. The scene is determined to involve macroshooting if the subject distance is less than or equal to a prescribedvalue.

Since there is a high possibility of blown-out highlights occurring whenflash shooting is performed at a short distance, desirably exposurecontrol is performed such that the flash is fired as little as possibleif the scene is determined to involve macro shooting. Also, in the caseof flash shooting being performed, settings may be configured such thattone characteristics that fill in high brightness portions are used bythe signal processing circuit 140, to ensure the tone of highlightportions does not readily become saturated.

FIGS. 13A and 13B are flowcharts illustrating operation of the actualimage capturing process in the image capturing apparatus of the presentembodiment.

Note that it is assumed that, during standing by to capture a stillimage, the system controller 107 determines the amount of dynamic rangeexpansion from EVF images at regular intervals, for example, using anabovementioned method. In the present embodiment, the amount of dynamicrange expansion (amount by which an AE target value is decreased) can bedetermined with the four steps from 0/3 steps to 3/3 steps in 1/3 stepincrements. Note that the range and size per step of the amount ofdynamic range expansion can be set arbitrarily. Also, when the shutterbutton included in the operation unit 112 is pressed a half stroke andan image capture preparation instruction is input, for example, thesystem controller 107 executes the abovementioned scene determinationprocess, using EVF images and the results of AF control and AE control.

In response to the shutter button included in the operation unit 112being pressed a full stroke and an image capture start instruction beinginput, during standing by to capture a still image, the systemcontroller 107 then starts the following processing.

At S400, the system controller 107 judges whether it was determined inthe scene determination that the scene is a sunset scene. If determinedto be a sunset scene, the system controller 107, at S408, computes theexposure correction amount for a sunset scene, and does not compute theamount of dynamic range expansion. On the other hand, if not determinedto be a sunset scene, the system controller 107, at S401, acquires, fromthe buffer memory 103, the amount of dynamic range expansion determinedimmediately before the image capture start instruction was input.

At S402, the system controller 107 determines whether the acquiredamount of dynamic range expansion, that is, the amount by which the AEtarget value is to be decreased can be realized by sensitivityadjustment in the AFE circuit 150 (control of CDS gain circuit and VGAgain circuit). This determination can be performed by comparing therange of sensitivity adjustable by the AFE circuit 150 and the amount ofdynamic range expansion acquired at S401. In the case where thedecreasing of sensitivity (decreasing of gain) equivalent to the amountof dynamic range expansion cannot be performed, the system controller107 judges that the amount of dynamic range expansion cannot be realizedwith only sensitivity adjustment by the AFE circuit 150.

In the case where the amount of dynamic range expansion can be realizedwith sensitivity adjustment in the AFE circuit 150, the systemcontroller 107 computes, at S404, the gain setting as an image capturecondition. Note that there are no particular restrictions on the way inwhich the CDS gain and VGA gain are combined in the setting, and settingis possible with an arbitrary combination.

On the other hand, if judged that the amount of dynamic range expansioncannot be realized with only sensitivity adjustment in the AFE circuit150, the system controller 107 changes an exposure condition, based onthe insufficient amount of gain that will remain even if available gaincontrol is performed by the AFE circuit 150 (S405). Specifically, thesystem controller 107 computes the exposure correction amount forrealizing the insufficient amount of gain.

Exposure correction here is minus correction, and can be realized by acommon method, such as reducing the aperture, increasing the shutterspeed, or inserting a neutral density filter (ND filter).

At S406, the system controller 107 determines whether the exposurecorrection computed at S405 is possible. For example, with an imagecapturing apparatus that does not have a ND filter, minus correction ofthe exposure cannot be performed in the case where the highest shutterspeed and the smallest aperture (maximum aperture value) have alreadybeen set by auto exposure control. Also, the shutter speed cannot beraised in the case where the highest settable shutter speed has beenset, when performing flash image capture. The same applies in cases suchas where the maximum shutter speed has been determined. Note thatbecause it is desirable not to change an aperture value set by the userif in aperture priority AE mode, it may be determined that minuscorrection is not possible if the shutter speed is already at thehighest setting. The same applies if in shutter priority AE mode.

If determined that minus correction of exposure equivalent to theinsufficient amount of gain remaining after gain adjustment is notpossible, the system controller 107, at S407, revises the amount ofdynamic range expansion to the maximum value realizable by sensitivityadjustment and exposure correction. The system controller 107 thencomputes the gain amount to be set in the AFE circuit 150, and furthercomputes the exposure correction amount as required.

At S409, the system controller 107 sets the gain amount as an imagecapture condition in the CDS gain circuit 152 and the VGA gain circuit154 of the AFE circuit 150. Also, in the case of performing exposurecorrection, the system controller 107 changes exposure parameters thatdepend on the AE result (settings such as shutter speed, aperture, NDfilter use) according to the amount of exposure correction, and sets thechanged exposure parameters in the lens apparatus 200 as image captureconditions.

At S410, the system controller 107 performs still image capture (actualexposure).

At S411, the system controller 107 determines whether the dynamic rangehas been expanded in processing prior to the actual exposure. Here, thesystem controller 107 determines that the dynamic range has not beenexpanded, if the amount of dynamic range expansion is 0 steps or if thescene was determined to be a sunset scene at S400.

At S412, the system controller 107 judges whether it was determined inthe scene determination process prior to the actual exposure that thescene is a night scene. If determined to be a night scene, the systemcontroller 107, at S415, selects tone characteristics for a night scenethat depend on the amount of dynamic range expansion. If not determinedto be a night scene, the system controller 107, at S413, computes ahistogram from the image for actual exposure, and, at S414, determinesthe amount of dynamic range contraction (D− amount).

FIGS. 14A and 14B show an example of tone characteristics selected whenthe scene is determined to be a night scene. As illustrated, noise indark portions can be made less prominent, and an appropriate image withsharp shadows is obtained as an image for when shooting a night scene,by flattening the characteristics of dark portions relative to normaltone characteristics (rate of increase of output luminance values islowered relative to the rate of increase of input luminance values).

Of course, these tone characteristics may be changed depending onsensitivity, aperture, zoom position and the like. For example, thedegree to which shadows are filled in may be reduced, since theperipheral brightness of the lens tends to fall when the aperture isreleased.

At S416, the system controller 107 sets, in the signal processingcircuit 140, tone characteristics that depend on the amount of dynamicrange expansion (excluding the case where the expansion amount is zeroor the scene is determined to be a sunset scene), the tonecharacteristics for a night scene selected at S415, or tonecharacteristics that depend on the amount of dynamic range contractiondetermined at S414.

FIG. 11A represents a conceptual view of the dynamic range in thepresent embodiment.

In the present embodiment, the dynamic range is defined as the ratio ofthe saturation signal amount luminance of the image sensor to correctluminance. Correct luminance is a luminance target value level for whenperforming automatic exposure control (AE), and is equivalent to anaverage value of screen luminance if the AE mode is an average meteringmode, for example.

Accordingly, the dynamic range can be defined as follows:dynamic range=sensor saturation signal amount luminance/AE target value

Note that the AE target value here is based on an output signal from theimage sensor 101 prior to sensitivity adjustment being performed by theAFE circuit 150.

The AE target value may be changed according to the AE mode, and even inthe case of there being an evaluative metering mode and a spot meteringmode, an AE target value for each mode can be used.

FIG. 11B shows an example of the relation between AE target value,saturation signal value and dynamic range.

It is clear from FIG. 11B that the dynamic range can be increased bylowering the AE target value.

FIGS. 12A and 12B show an example setting of tone characteristics in thesignal processing circuit 140 of the present embodiment.

An example setting of tone characteristics (brightness correctionamount) in the case where the amount of dynamic range expansion is setto the four steps of normal (0/3 steps), +1/3 steps, +2/3 steps, and+3/3 steps is shown.

Here, the AE target values corresponding to respective amounts ofdynamic range expansion are the same as those shown in FIG. 11B. Asshown in FIGS. 12A and 12B, the tone characteristics are set such thatthe AE target value after performing tone correction on the AE targetvalue for each expansion of dynamic range will be a normal AE targetvalue at which the dynamic range is not expanded, irrespective of theamount of dynamic range expansion.

As described using FIG. 11A and FIG. 11B, the dynamic range can beexpanded by lowering the AE target value. However, simply lowering theAE target value results in underexposure and captured images will bedark. For this reason, the dynamic range can be expanded while correctlysetting the brightness (exposure) of captured images, by performing tonecorrection in the signal processing circuit 140 so as to brighten thecaptured image data, according to the amount of dynamic range expansion.

Note that in the present embodiment, a configuration is illustrated inwhich a drop in the luminance of a captured image due to the AE targetvalue being lowered is compensated by tone correction, but similarluminance correction may be performed using different means such as alookup table.

Gain such as the gain of white balance coefficients for correcting whitebalance, and the clipping amount for determining the saturation signalamount may also be controlled. In other words, the same effect isobtained even if gain is increased by a downstream signal processingcircuit, and the clipping amount is expanded (saturation signal amountincreased) by the amount of the increase in gain, after A/D conversionhas been performed on an image signal whose gain has been decreased by adecrease in the amount of exposure or a decrease in the AFE gain.

Here, the relation between the sensitivity setting (gain setting of theAFE circuit 150) and the saturation of the image sensor 101 will bedescribed.

Generally, gain applied to the output signal of an image sensor iscontrolled, such that output satisfies a prescribed relation withrespect to the amount of light and the exposure value of the camera(input).

However, there is an upper limit to the amount of charge that can bestored in the photodiodes of an image sensor. For this reason, when thegain applied by the AFE circuit 150 is lowered with the object oflowering the ISO speed of the image sensor, the maximum signal amountafter the gain has been applied also drops. Accordingly, the saturationsignal amount also drops together with the drop in gain.

Thus, configuring a desired sensitivity setting is possible disregardingamplification of noise, with regard to increasing the sensitivity of theimage sensor, whereas a limit value arising from the saturation signalamount exists with regard to decreasing sensitivity.

In S409 of FIG. 13B, in the case where sensitivity cannot be lowered, itis often the case that the minimum settable sensitivity has already beenset in the image capturing apparatus. This means that the gain appliedto the output signal of the image sensor has already been decreased to avalue equivalent to the minimum sensitivity. For this reason,sensitivity cannot be further decreased by controlling the gain of theAFE circuit 150. Accordingly, in the present embodiment, a furtherdecrease in sensitivity is realized by exposure correction, in the casewhere the target amount by which the AE target value (sensitivity) is tobe decreased cannot be realized by controlling the gain of the AFEcircuit 150.

The operation for performing tone correction in the signal processingcircuit 140 to correct brightness with respect to a dark image obtainedwhen image capture is performed at a sensitivity decreased by exposurecontrol is ultimately the same as increasing sensitivity, and producesdegradation in image quality due to noise also being amplified when tonecorrection is performed.

However, in the present embodiment, decreasing the ISO speed with gaincontrol is preferentially performed, in the case where decreasing of ISOspeed corresponding to the amount of dynamic range expansion can berealized by the gain control of the AFE circuit 150. In the case where adecrease in ISO speed corresponding to the amount of dynamic rangeexpansion cannot be realized only by gain control, the gain is decreasedto a minimum value and the amount by which the decrease in sensitivityis deficient is made up for with exposure correction. In this case,noise itself is not significant, even if amplified when tone correctionis performed, since the gain of the AFE circuit 150 has already beendecreased to a level equivalent to the minimum sensitivity. For thisreason, degradation in image quality can be minimized.

FIG. 15 is a flowchart illustrating operation of determining an amountof dynamic range contraction (D−) in the image capturing apparatusaccording to the embodiment of the present invention. The operationshown in FIG. 15 corresponds to the operation of acquiring a histogramat S413 and determining the amount of dynamic range contraction at S414in FIG. 13B.

At S501, the system controller 107 multiplies the white balancecoefficients (WB coefficients) for each of the colors R, G and B withrespect to CCD-RAW image data stored in the buffer memory 103. The WBcoefficients used here may be computed from an image resulting fromactual image capture or may be computed from the EVF image immediatelypreceding actual image capture. Note that in the latter case, WBcoefficients that depend on the color temperature of flash light or theWB coefficients of a D55 light source or daylight can be used in thecase of flash shooting with actual image capture.

Multiplying WB coefficients enables the saturation amount for each colorto be accurately detected, and enables detection even with a subjectwith respect to which a specific color is saturated.

At S502, the system controller 107 acquires a cumulative histogram foreach color relating to the image obtained by multiplying the WBcoefficients, using the histogram creation circuit 130.

At S503, the system controller 107 acquires, for each color as ahighlight luminance value, a luminance value Y_Hp at which the frequencyof the cumulative histogram is a prescribed value (e.g., 80%).

At S504, the system controller 107 computes the gain value Gain_yhp withrespect to a predetermined saturation target value Sat_TargetY for eachcolor with the following equation:Gain_(—) yhp=Sat_TargetY/Y _(—) hp

The system controller 107 then sets the smallest gain value out of theGain_yhp values computed for each color as the gain value Gain_yhpcorresponding to the final amount of dynamic range contraction.

At S505, the system controller 107 selects tone characteristicparameters corresponding to the gain value computed at S504.

FIG. 16 shows an example of the relation between final amounts of gainincrease that depend on the final value of the gain value Gain_yhpcomputed at S504 and ISO speed.

In the present embodiment, there are four final amounts of gainincrease, namely, 0, 1.12, 1.25 and 1.41, and the system controller 107respectively selects the following:

-   -   1.12→tone characteristic equivalent to a 1/6 step increase in        exposure    -   1.25→tone characteristic equivalent to a 1/3 step increase in        exposure    -   1.41→tone characteristic equivalent to a 1/2 step increase in        exposure

FIGS. 17A and 17B show the relation between amount of gain increase andtone characteristic setting value. Increasing exposure 1/6 steps is thesame as increasing the AE target value 1/6 steps. The dynamic rangecontraction process thus corrects the tone characteristics of an imageobtained by the actual exposure, and lightens the image by a number ofsteps that depends on the final amount of gain increase.

In the present embodiment, the amount of dynamic range contraction isvaried by varying the tone characteristic setting, but the amount ofdynamic range contraction can also be varied by controlling the WBcoefficients using a lookup table or the like.

Also, in the present embodiment, histogram analysis is performed on theRAW image when the actual exposure is performed, and the amount ofdynamic range contraction is determined. However, exposure control maybe performed after performing histogram analysis on EVF images anddetermining the amount of dynamic range contraction, similarly todetermining the amount of dynamic range expansion. In this case, becausegain does not need to be increased using tone characteristics,contraction of dynamic range is possible without noise amplification dueto the gain increase.

However, there are also cases where the extent of the blown-outhighlights of an image cannot be judged until the actual exposure isperformed, such as with flash shooting, for example. For this reason,exposure control may be performed after determining the amount ofdynamic range contraction based on EVF images when the flash is notfired, and using the result of analyzing images obtained by the actualexposure when the flash is fired.

FIG. 18 shows an example of the relation between the setting of exposurecontrol, dynamic range expansion process (D+), dynamic range contractionprocess (D−), and tone characteristics for a night scene, according tothe scene determination result, in the image capturing apparatusaccording to the embodiment of the present invention. Note that, in thepresent embodiment, the dynamic range contraction process is performedif the dynamic range has not been expanded. Accordingly, in FIG. 18, thedenoting of ◯ (or ⊚) for both D+ and D− indicates that execution of D+or D− is possible, and does not indicate that both are performed.

If the scene is determined to be a night scene and people are notincluded (face not detected by the face detection circuit 120), adynamic range expansion process tailored to the amount of dynamic rangeexpansion determined based on EVF images is performed. Also, asabovementioned, tone characteristics that fill in dark parts areselected.

On the other hand, since the area of the dark portion is large in thecase of a night scene, tone characteristics that increase sensitivityare selected when performing the dynamic range contraction process usingthe luminance information of a histogram or the like. However, thedynamic range contraction process (D−) is not performed with a nightscene, since it is appropriate for a somewhat darker image to becaptured in comparison to a normal scene.

On the other hand, if the scene is determined to be a night scene andpeople are included, the dynamic range expansion or dynamic rangecontraction process is performed, and tone characteristics for a nightscene are not selected.

If people are included in a night scene, an appropriate image cannot beacquired when tone characteristics for a night scene are selected, sincecontrast on the face increases too much, and the tone characteristics ofdark parts of the face are flattened. Also, flash shooting is oftenperformed if people are included in a night scene, but the flash lightoften does not adequately reach the subject in the case of the subjectdistance being long, as in the case where image capture is performedwith the telescopic end of a zoom lens. For this reason, performing thedynamic range contraction process enables an appropriate image to beacquired with regard to the face region of people and the like.

If the scene is determined to be a sunset scene and people are notincluded, exposure control is performed such that the image will beunderexposed relative to when normal shooting is performed in the actualexposure, and the dynamic range expansion process is prohibited.

With a scene determined to be a sunset scene, the sun is often in theangle of view. For this reason, the effect of further performing minuscorrection and expanding the dynamic range is muted, since the result ofauto exposure control tends toward underexposure. Rather, the dynamicrange expansion process (D+) may increase the ISO speed, in which casethe possibility increases of noise being amplified, and an inappropriateimage resulting. Also, saturation is controlled so that red is enhanced.

Also, the brightness of the face cannot be controlled if the flash isnot fired in the case where people are included in a sunset scene.Accordingly, an appropriate image can be acquired by performing thedynamic range expansion process (D+) or the dynamic range contractionprocess (D−), together with implementing exposure determination withpriority for the face (face priority AE).

On the other hand, if people are included in a sunset scene and theflash will be fired, the brightness of the face can be controlled bycontrolling the flash metering. For this reason, an appropriate imagecan be acquired by underexposing when image capture is performed, andprohibiting the dynamic range expansion process, similarly to the casewhere people are not included. Also, in the case where people areincluded in a sunset scene, further enhancing of red is not performed,so as to not affect the skin tones.

If the scene is determined to be a backlight scene and people are notincluded, the scene will often be sufficiently bright. In other words,it will often be the case that a low ISO speed and a fast shutter speedhave been set by the image capturing apparatus. As a result, there is ahigh possibility of not being able to perform the dynamic rangeexpansion process. If people are not included and the main subject is alandscape, an image with little noise will often be more appropriate.

On the other hand, in the case where people are included, the peoplewill often be in the shade or in a dark place in the room, and a sunnyplace or an outdoor area will form the background. Thus, in the casewhere, for instance, a different light source is mixed in a singleshooting scene, the amount of dynamic range expansion will often not besufficient at 1 step. Also, in cases such as a snap shot, shooting theface and the background is more appropriate.

Thus, with a backlight scene, a better image can be shot by increasingthe amount of dynamic range expansion in the case of people beingincluded, and not increasing the amount of dynamic range expansion inthe case of people not being included. For example, with a normal scenedetermination result, the amount of dynamic range expansion is increasedto a maximum of 1 step, whereas in the case where people are included ina backlight scene, the amount of dynamic range expansion is extended to2 steps, for example.

Of course, the upper limit of the expansion amount determined in thedynamic range expansion process may, needless to say, be determinedaccording to the degree to which ISO speed is raised. For example,control can be performed to increase the amount of dynamic rangeexpansion in the case of a brightness level for shooting at a highsensitivity, and to decrease the amount of dynamic range expansion inother cases.

In the case of the scene being determined to be a blue sky scene,processing is similar to a normal scene, apart from enhancing bluesaturation.

In the case where, however, the scene is determined to be a blue skyscene, and there are many blocks in the cyan hue direction with highluminance in blue portions, the upper limit of the amount of dynamicrange expansion may be extended. This enables the amount of dynamicrange expansion to be determined while actively determining the degreeof blown-out highlights in sky portions.

Also, in the case of a portrait scene, the dynamic range expansion orcontraction process is performed together with person priority AE.

In the case of a normal scene (i.e., not a night, sunset or backlightscene, and where people are not detected), an increase in noise can besuppressed by performing the dynamic range expansion process in a rangein which the ISO speed remains unchanged, and only performing thedynamic range contraction process at a low ISO speed.

In the case where the scene is judged to be a blue sky scene (withoutpeople), processing is similar to a normal scene, apart from enhancingblue saturation.

In the case where, however, the scene is judged to be a blue sky scene,and there are many blocks in the cyan hue direction with high luminancein blue portions, the upper limit of the amount of dynamic rangeexpansion may be extended. This enables the amount of dynamic rangeexpansion to be determined while actively determining the degree ofblown-out highlights in blue portions.

Other Embodiments

In the abovementioned embodiment, the case was described where dynamicrange expansion by the present invention is applied in still imagecapture, but can be similarly applied when performing EVF image captureor moving image capture. In this case, the timing at which theparameters are set is adjusted such that the gain control (and exposurecorrection as required) of the AFE circuit 150 and tone correction inthe signal processing circuit 140 are applied to the same image.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2009-34392, filed on Feb. 17, 2009, No. 2009-26694, filed on Feb. 6,2009, and No. 2009-26695, filed on Feb. 6, 2009, which are herebyincorporated by reference herein in their entirety.

What is claimed is:
 1. An image capturing apparatus comprising: a scenedetermining unit arranged to perform scene determination, based oninformation regarding a moving image captured prior to the capture of astill image; a gain control unit arranged to control a gain value basedon an output from the scene determination unit; and a control unitarranged to control the image capturing apparatus to capture an image,and to selectively execute, based on the output from the scenedetermination unit, one of (a) a first process for capturing a stillimage at a decreased gain value, and applying, with respect to acaptured still image, tone correction for compensating the decreasedgain value, and (b) a second process for capturing a still image, andapplying, with respect to a captured still image, tone correction forincreasing the brightness of a low luminance portion of the capturedstill image, based on a characteristic of a high luminance portion ofthe moving image or the captured still image.
 2. The image capturingapparatus according to claim 1, wherein the scene determining unitdetermines at least one of: a presence of a person, a night scene, ablue sky scene, a sunset scene, a backlight scene, and macro shooting.3. The image capturing apparatus according to claim 1, wherein thecontrol unit is arranged to execute the first process, in a case wherethe result of the scene determination does not indicate a predeterminedscene on which the first process is not to be performed.
 4. The imagecapturing apparatus according to claim 1, Wherein the control unit isarranged to execute the second process, in a case where the result ofthe scene determination indicates a predetermined scene on which thefirst process is not to be performed, and in a case where an amount ofthe decreased gain value in the first process is zero.
 5. The imagecapturing apparatus according to claim 1, wherein in the first process,the greater an amount of blown-out highlights in an image on which thescene determination was performed, the greater an amount of thedecreased gain value is determined by the control unit.
 6. The imagecapturing apparatus according to claim 1, wherein the control unit, inthe second process, is arranged to apply, to the captured still image,tone correction such that a luminance value at which a frequency of acumulative histogram of the captured still image is a prescribed valuewill be a luminance value corresponding to a saturation signal amount ofan image sensor included in the image capturing apparatus.
 7. The imagecapturing apparatus according to claim 1, wherein, in a case where ascene is determined in the scene determination to be a night scene, thecontrol unit is arranged to execute a third process of applying tonecorrection for decreasing a rate of increase in input-outputcharacteristics in the low luminance portion.
 8. An image capturingapparatus comprising: a scene determining unit arranged to perform scenedetermination before the capture of a still image; a gain control unitarranged to control a gain value based on an output from the scenedetermination unit; and a control unit arranged to control the imagecapturing apparatus to capture an image, and to selectively execute,based on the output from the scene determination unit, one of (a) afirst process for capturing a still image at a decreased gain value, andapplying, with respect to a captured still image, tone correction forcompensating the decreased gain value, and (b) a second process forcapturing a still image, and applying, with respect to a captured stillimage, tone correction for increasing the brightness of a low luminanceportion of the captured still image, based on a characteristic of a highluminance portion of a moving image or the captured still image.
 9. Acontrol method of an image capturing apparatus, comprising: a scenedetermining step of performing scene determination, based on a movingimage captured prior to the capture of a still image; a gain controlstep of controlling a gain value based on a result of the scenedetermination in the scene determining step; and a control step ofcontrolling an image capturing operation, wherein in the control step,based on the result of the scene determination in the scene determiningstep, one of (a) a first process for capturing a still image at adecreased gain value, and applying, with respect to a captured stillimage, tone correction for compensating the decreased gain value, and(b) a second process for capturing a still image, and applying, withrespect to a captured still image, tone correction for increasing thebrightness of a low luminance portion of the captured still image, basedon a characteristic of a high luminance portion of the moving image orthe captured still image, is selectively executed.
 10. A control methodof an image capturing apparatus, comprising: a scene determining step ofperforming scene determination before the capture of a still image; again control step of controlling a gain value based on a result of thescene determination in the scene determination step; and a control stepof controlling an image capturing operation, wherein in the controlstep, based on the result of the scene determination in the scenedetermining step, one of (a) a first process for capturing a still imageat a decreased gain value, and applying, with respect to a capturedstill image, tone correction for compensating the decreased gain value,and (b) a second process for capturing a still image, and applying, withrespect to a captured still image, tone correction for increasing thebrightness of a low luminance portion of the captured still image, basedon a characteristic of a high luminance portion of a moving image or thecaptured still image, is selectively executed.
 11. A non-transitorycomputer-readable recording medium having recorded thereon a program forcausing a computer of an image capturing apparatus to execute the stepsof the control method claimed in claim
 9. 12. A non-transitorycomputer-readable recording medium having recorded thereon a program forcausing a computer of an image capturing apparatus to execute the stepsof the control method claimed in claim
 10. 13. An image capturingapparatus comprising: a scene determining unit arranged to perform scenedetermination; and a control unit arranged to control the imagecapturing apparatus to selectively execute, based on an output from thescene determination unit, one of a first process and a second process,wherein said first process includes a tone correction process forincreasing a ratio of an input luminance value corresponding to themaximum output luminance value to a luminance target value used forautomatic exposure control to be larger than a reference ratio, and saidsecond process includes a tone correction process for decreasing theratio to be less than the reference ratio.
 14. The image capturingapparatus according to claim 13, wherein the control unit executes thetone correction according to an increased amount of the ratio in thefirst process.
 15. The image capturing apparatus according to claim 13,wherein the control unit executes the tone correction according to adecreased amount of the ratio in the second process.
 16. The imagecapturing apparatus according to claim 13, wherein, in the firstprocess, the control unit increases the ratio to be larger than thereference ratio by decreasing the luminance target value to be less thana reference luminance target value.
 17. The image capturing apparatusaccording to claim 14, wherein, in the first process, the control unitadjusts an exposure for when performing an actual image captureaccording to the decreased amount of the luminance target value.
 18. Theimage capturing apparatus according to claim 14, wherein, in the firstprocess, the control unit adjusts a gain value for when performing anactual image capture according to the decreased amount of the luminancetarget value.
 19. A control method of an image capturing apparatuscomprising: scene determining step of performing scene determination;and control step of controlling the image capturing apparatus toselectively execute, based on a determination result of the scenedetermination, one of a first process and a second process, wherein saidfirst process includes a tone correction process for increasing a ratioof an input luminance value corresponding to the maximum outputluminance value to a luminance target value used for automatic exposurecontrol to be larger than a reference ratio, and said second processincludes a tone correction process for decreasing the ratio to be lessthan the reference ratio.
 20. A non-transitory computer-readablerecording medium having recorded thereon a program for causing acomputer of an image capturing apparatus to execute the steps of thecontrol method claimed in claim
 19. 21. An image capturing apparatuscomprising: a control unit arranged to control the image capturingapparatus to selectively execute one of a first process and a secondprocess for a captured image based on a scene, wherein said firstprocess includes a tone correction process for applying a bigger gain tosignal of a low luminance portion than a gain applied to signal of ahigh luminance portion based on an exposure of the captured image, andsaid second process includes a tone correction process using a tonecharacteristics, which specifies a relation between input luminancevalues and corresponding output luminance values, and of which an inputluminance value associated with a maximum output luminance value ischanged based on the captured image.
 22. A control method of an imagecapturing apparatus comprising: control step of controlling the imagecapturing apparatus to selectively execute one of a first process and asecond process for a captured image based on a scene, wherein said firstprocess includes a tone correction process for applying a bigger gain tosignal of a low luminance portion than a gain applied to signal of ahigh luminance portion based on an exposure of the captured image, andsaid second process includes a tone correction process using a tonecharacteristics, which specifies a relation between input luminancevalues and corresponding output luminance values, and of which an inputluminance value associated with a maximum output luminance value ischanged based on the captured image.
 23. A non-transitorycomputer-readable recording medium having recorded thereon a program forcausing a computer of an image capturing apparatus to execute the stepsof the control method claimed in claim 22.